Tuning L1-SVM Hyperparameters with Modified Radius Margin Bounds and Simulated Annealing
نویسندگان
چکیده
In the design of support vector machines an important step is to select the optimal hyperparameters. One of the most used estimators of the performance is the Radius-Margin bound. Some modifications of this bound have been made to adapt it to soft margin problems, giving a convex optimization problem for the L2 soft margin formulation. However, it is still interesting to consider the L1 case due to the reduction in the support vector number. There have been some proposals to adapt the Radius-Margin bound to the L1 case, but the use of gradient descent to test them is not possible in some of them because these bounds are not differentiable. In this work we propose to use simulated annealing as a method to find the optimal hyperparameters when the bounds are not differentiable, have multiple local minima or the kernel is not differentiable with respect to its hyperparameters.
منابع مشابه
A Quadratic Loss Multi-Class SVM for which a Radius-Margin Bound Applies
To set the values of the hyperparameters of a support vector machine (SVM), the method of choice is cross-validation. Several upper bounds on the leave-one-out error of the pattern recognition SVM have been derived. One of the most popular is the radius–margin bound. It applies to the hard margin machine, and, by extension, to the 2-norm SVM. In this article, we introduce the first quadratic lo...
متن کاملEvaluation of simple performance measures for tuning SVM hyperparameters
Choosing optimal hyperparameters for support vector machines is an important step in SVM design. This is usually done by minimizing either an estimate of generalization error or some other related performance measures. In this paper, we empirically study the usefulness of several simple performance measures that are very inexpensive to compute. The results point out which of these performance m...
متن کاملRadius-margin Bound on the Leave-one- out Error of the Llw-m-svm
To set the values of the hyperparameters of a support vector machine (SVM), one can use cross-validation. Its leave-one-out variant produces an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirement. To overcome this difficulty, several upper bounds on the leave-one-out error of the pattern recognition SVM have been derived. The most pop...
متن کاملRadius-Margin Bound on the Leave-One-Out Error of a M-SVM
Using a support vector machine (SVM) requires to set the values of two types of hyperparameters: the soft margin parameter C and the parameters of the kernel. To perform this model selection task, the method of choice is cross-validation. Its leave-one-out variant is known to produce an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirem...
متن کاملA Quadratic Loss Multi-Class SVM
Using a support vector machine requires to set two types of hyperparameters: the soft margin parameter C and the parameters of the kernel. To perform this model selection task, the method of choice is cross-validation. Its leave-one-out variant is known to produce an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirement. To overcome thi...
متن کامل